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for the evaluation of university teaching 


Abstract 

The use of evaluation to examine and improve the quality of teaching and courses is now a component of most 
universities. However, despite the various methods and opportunities for evaluation, a lack of understanding 
of the processes, measures and value are some of the major impediments to effective evaluation. Evaluation 
requires an understanding of what to evaluate, how to evaluate it, how to collect and analyse the information 
and how to action and implement what has been learned. The aim of this paper is to describe an instrument to 
document not only the evaluative process, but also evaluative outcomes. The Course Improvement Flowchart 
(CIF) provides a template for the application of an action inquiry framework. Throughout this process, the 
CIF template can be used to document key recommendations emanating from multiple sources of feedback so 
that a proactive statement of intent or action plan for teaching and course improvement is written. The 
resultant improvements that maybe achieved in teaching and course quality may lead to more satisfying 
teaching experiences, improved student outcomes and heightened professional growth as a teacher. 
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Abstract 

The use of evaluation to examine and improve the quality of teaching and courses is now a 
component of most universities. However, despite the various methods and opportunities for 
evaluation, a lack of understanding of the processes, measures and value are some of the 
major impediments to effective evaluation. Evaluation requires an understanding of what to 
evaluate, how to evaluate it, how to collect and analyse the information and how to action and 
implement what has been learned. The aim of this paper is to describe an instrument to 
document not only the evaluative process, but also evaluative outcomes. The Course 
Improvement Flowchart (CIF) provides a template for the application of an action inquiry 
framework. Throughout this process, the CIF template can be used to document key 
recommendations emanating from multiple sources of feedback so that a proactive statement of 
intent or action plan for teaching and course improvement is written. The resultant 
improvements that may be achieved in teaching and course quality may lead to more satisfying 
teaching experiences, improved student outcomes and heightened professional growth as a 
teacher. 
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The Context of Evaluation and Quality 
Assurance at Australian Universities 


There has been an increasing focus in Australian higher education on performance and measuring 
output and outcomes. Greater attention is being paid to the evaluation of policies and practice through 
quality assurance assessment by the Australian Universities Quality Agency (AUQA) (Sharma, 2004). 
AUQA conducts ‘quality’ audits of universities that are based on judging whether university procedures 
match objectives at an institutional level rather than against pre-determined standards. AUQA 
examines how universities attempt to achieve their goals and whether they have been achieved. 
AUQA reports on performance and outcomes (e.g. student progress, achievement and graduate 
attributes) to enhance the quality of universities’ academic activities (AUQA, 2007). 

Teaching performance is a key component of the quality assurance process and can be measured in 
terms of scores on the Course Experience Questionnaire (CEQ) and success in the Learning and 
Teaching Performance Fund (LTPF). The LTPF was introduced in 2003 by the Australian Government 
to reward universities that can demonstrate excellence in learning and teaching. Universities are 
asked to submit evidence that their policies and practices relating to learning and teaching, as well as 
student evaluations of subjects, are publicly available on their websites. Key performance indicators 
include student satisfaction, student outcomes and success. Student satisfaction indicators are based 
on the responses to the generic skills scale, good teaching scale, and overall satisfaction items from 
the CEQ. Notably, the focus on teaching quality in higher education in the last 15 years has resulted in 
heightened awareness and interest regarding strategies to evaluate teaching effectiveness. 

Evaluation of University Teaching 

Evaluation of teaching has been described as a vehicle for the improvement of teaching (1986, 
Saroyan & Amundsen, 2001). Evaluation may facilitate improvement in the quality of teaching but also 
can focus on other elements such as courses and administration (Hughes, 2002). Evaluation can 
facilitate change and improvement through reflective practice and provide an indication of course or 
teaching effectiveness. 

Hughes (2002) outlined the many purposes of evaluation in the university context including formative 
(quality improvement) and summative (quality assurance) methods, although these distinctions often 
overlap. In quality improvement, university teachers may look to identify aspects of courses and 
teaching that are effective, explore suggestions for improvement to modify teaching practices and 
support various processes in performance reviews. In these instances, evaluation can inform how well 
a teacher is teaching but also what aspects of teaching are good and what aspects need 
improvement. While higher education institutions contain both good and bad teachers, there is also 
variation in each teacher's potential to improve over time. Good teachers continually improve and are 
constantly searching for strategies to ensure the quality of courses they present to students are high. 
In quality assurance, various evaluative processes can inform promotion, teaching awards, 
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performance reviews and support claims of teaching excellence. Promotion within universities often 
requires teaching portfolios and descriptions of one's teaching practices. Evaluation is a key 
component of any teaching portfolio to demonstrate a commitment to quality teaching. 

The use of evaluation to examine and improve the quality of teaching is increasingly common at 
universities. This change is largely attributable to initiatives such as AUQA and the LTPF which focus 
on measuring and rewarding performance and outcomes related to teaching. However, an 
understanding of, and commitment to appropriate evaluation procedures is often lacking within the 
university setting (Hughes, 2002). Despite the various methods and opportunities for evaluation, the 
realities of academic life are that circumstances may not exist for this to occur effectively. Moreover, a 
lack of accountability and motivation may exist if evaluation is to be instigated by individual teachers 
who may not realise or believe in its value. Overall, poor levels of understanding of the processes, 
measures and value are some of the major impediments to effective evaluation being undertaken in 
universities. 

The Process of Evaluation 


The process of evaluation is not straightforward as teaching is an inherently complex, multifaceted and 
personal experience. Evaluation requires an understanding of what to evaluate, how to evaluate it, 
how to collect and analyse the information and how to action and implement what has been learned. 
Planning for evaluation requires thinking about the aims of the evaluation, the context, who will be 
involved, what sources will be used and the outcomes (McCormack, 2003). 

In many universities contexts, problems may exist as summative evaluation is given far greater 
emphasis. The recent shift in universities toward funding practices and quality assurance has meant 
student evaluation instruments are exclusively viewed as measures of success and status (Hughes, 
2002). Given summative student evaluations have been found to be one of the most common sources 
of feedback regarding university teaching (Saroyan & Amundsen, 2001; Marsh, 1987), there is a 
danger that they are over used and become the sole basis for judging teaching effectiveness. 
Furthermore, collecting data on teaching strengths for promotion purposes may take priority over 
examining areas of teaching that need improvement (Hughes, 2002). It is important that teaching 
evaluation at university extends beyond summative student questionnaires. Informed judgements 
about effectiveness of teaching are made when multiple perspectives in a range of teaching 
characteristics are sought. The information also needs to be sensitive to the context of teaching. 
Teaching contexts may vary in terms of disciplines, level of the course, teaching styles, philosophies 
and course objectives. 

In terms of evaluation, one single data source (e.g. students, tutors or peers) and one single collection 
method (e.g. questionnaires, interviews, discussions) may reveal only one perspective of teaching. 
Evidently, several different sources and methods should be utilised to provide a more comprehensive 
overview and to ensure a range of teaching processes are explored (Saroyan & Amundsen, 2001). 

Each of these sources can provide unique information but in isolation do have some limitations. That 

tl 


Journal of University Teaching and Learning Practice - Vol 5/2, 2008 




The Course Improvement Flowchart: A description of a tool and process for the 

evaluation of university teaching 

Philip Morgan 


is, sources have strengths and weaknesses but the weakness of one source may be overcome via the 
strength of another. It is also important to recognise that evaluation can operate on a number of 
different levels and does not necessarily need to be focused on classroom practices. All dimensions of 
teaching (e.g. content knowledge, pedagogies etc.) and the course (e.g. assessment, lectures, 
materials, resources etc) need to be evaluated. Evaluation can also examine the design of courses 
and planning practices and the achievement of student outcomes. 

When multiple perspectives and techniques are used, the evaluative process will be a more valid and 
reliable indicator of teacher effectiveness and recommendations for change will be more credible and 
valuable. Despite the literature promoting the benefits of multiple methods of evaluation, many 
universities and teachers do not have access to or utilise an instrument or template to guide and 
document this process or add credibility and accountability to the evaluation outcomes. Figure 1 (see 
page 14) illustrates an example of the Course Improvement Flowchart ( CIF ) which has been 
developed as a tool to improve the quality of university courses and teaching. The CIF provides a 
framework for the collection of formative and summative evaluation data which encourage teachers to 
reflect on their teaching performance and engage in professional learning. The CIF systematically and 
explicitly displays the framework, flowchart and action plan for course improvement, which is based on 
a number of key evaluative processes. The CIF can be used to evaluate teaching and underscores the 
value of obtaining multiple sources of feedback when teaching and recording and acting upon 
information that has been collected. This process focuses on revising teaching practice and may 
inform decisions about future course development and delivery. 

The CIF is a process that recognises the purposes, teaching activities, outcomes and reflections to 
improve which is similar to the ADRI cycle (Approach; Deployment; Results; Improvement) endorsed 
and utilised by AUQA. The ADRI cycle is a widely recognised framework to approach evaluation and 
is known across the sector as a quality management model and espouses a quality improvement 
cycle. To this end, the ADRI cycle provides a framework for the evaluation of many aspects of 
university functioning, standards and practices. As Baird (2005) advised, the ADRI cycle refers to: 

- Approach - planned arrangements are suitable to achieve goals and measures of success and 
improvement are defined 

- Deployment - actual practice conforms to planned arrangements and improvement can be based on 
refinement to determine alternative strategies and not just success 

- Results - arrangements achieve the desired results and make recommendations for improvement 

- Improvement - learning occurs and results from evaluation are disseminated to key stakeholders. 

The ADRI is cyclical in nature and focuses on continuous improvement which aligns with the 
framework of the CIF. The ADRI and CIF process both provide a way of thinking about cycles of 
quality and are designed to be used at more than one time point. 
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Sources of Feedback 


As illustrated in Figure 1, the CIF includes feedback from students (numerical questionnaire scores, 
open-ended comments, focus group interviews, course evaluation results), colleagues (classroom 
observations, meetings, tutor course evaluation) and self-reflection (course evaluation, reflection 
statements). A brief description of each CIF feedback component is now provided: 

Student Evaluation of Teaching (SET) 

A fundamental component of improvement in teaching is student feedback. SET is the most common 
source of feedback regarding university teaching (Saroyan & Amundsen, 2001; Marsh, 1987) and 
provide an indication of student perceptions of the impact of teaching on their learning (Saroyan & 
Amundsen, 2001). However, many teachers and institutions do not necessarily value or encourage 
student feedback (Richardson, 2005) despite student evaluations generally being considered as 
reliable and valid (Paulsen, 2002). Appropriate use of student feedback can have a positive impact on 
teaching practices. In a review of the impact of evaluation on teaching, Murray (1997) concluded that 
student evaluation of teaching does lead to improvement of teaching. 

The use of student evaluations as a measure of teaching effectiveness has been a contentious issue 
in the literature (Shevlin et al, 2000; Saroyan & Amundsen, 2001). In a meta-analysis of SET, Cohen 
(1981) concluded that students’ ratings of instruction are a valid measure of instructional 
effectiveness. Marsh (1987) reviewed meta analyses and various research reports and concluded that 
student evaluations were relatively unbiased, valid and reliable, and provided a useful means of 
feedback. But some have argued that student evaluations don't fully reflect the effectiveness of 
teachers (Kember et al, 2002). Furthermore, research has illustrated that students can provide 
valuable feedback in some areas (rapport, workload, usefulness of materials, what they have learned, 
clarity of presentation, concern for progress and welfare) (McCormack, 2003) but may not be able to 
make accurate judgements in other instances (appropriateness of content, materials, course 
objectives and assessment of student work). 

Ultimately, student feedback needs to be viewed as one component of information regarding teaching. 

It is generally accepted that having respect for student perceptions of courses and teaching and other 
issues is at the core of good teaching (Taylor, 1995). Students are in a unique position to contribute 
useful feedback and effective teachers listen to what students say and are open to their ideas 
regarding how to improve their teaching (Brookfield, 1990). Students can provide the most useful 
information regarding the direct impact of teaching and the various processes of teaching. 

In a meta-analysis of research on the effects of university education, Pascarella & Terenzini (1998) 
strongly endorsed that learning is enhanced when students are closely engaged with all aspects of the 
course they are studying. It is therefore important to ensure that feedback instruments provide 
students with an opportunity to reflect on all aspects of their learning experiences and are not biased 
towards more traditional or teacher-centred approaches. For example, only asking students to reflect 
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on whether the teacher had adequate knowledge may not allow students to reflect on their total 
learning experience and thus should not be used exclusively to make judgements about the quality of 
teaching. The items used in surveys should represent a range of effective teaching methodologies and 
not just the ‘sage on the stage’ approach where the teacher is seen as the holder of all knowledge and 
the student acts to ‘listen and [passively] learn’ from the teacher. 

The CIF outlines three major types of student feedback of teaching which can be collected in 
quantitative and qualitative forms: (a) Numerical Questionnaire Scores, (b) Open-ended questions and 
(c) Focus Group Interviews. 

(a) Numerical Questionnaire Scores can be used to collect data on a range of teaching and course 
attributes and is one of the most common sources of student feedback. One advantage of 
questionnaires is that responses can be obtained by the whole student group, are anonymous and can 
provide longitudinal data on trends. Students can indicate how effective they believe the course and 
teaching have been. As displayed in the CIF (Figure 1), a SET questionnaire can be used as one 
source of feedback from the evaluative process. The results can be used for two purposes (i) to 
identify areas of strength and ways in which these might be developed further, (ii) to identify areas that 
may be improved. Data should be collected from a range of teaching attributes: i.e. the course, 
studying material in the course, lectures in the course, lecturing, tutorials, students’ views of outcomes 
in the course, assessment, and resources inside and outside the classroom. The interpretation of the 
results is dependent upon the nature of the course. Although numerical data represent the ratings 
given by the students to various aspects of teaching, they should be considered as a means to identify 
trends such as changes in student perceptions. 

(b) Open-ended questions: A key component of student feedback is open-ended questions which can 
be included as part of the course survey. Open-ended questions are a valuable source of students' 
ideas about improving courses using questions such as 'What are the best things about the course?' 
Flow could the course be improved? By reading all open-ended comments and searching for important 
themes, one can obtain a valuable measure of students' perceptions and satisfaction with the course. 
These questions may enrich quantitative data and allow students an opportunity to list positive and 
negative aspects of the course. Weaknesses of the course are often identified and suggestions useful 
in trying to determine how to improve courses and teaching. The quality of information in terms of 
modifying courses and teaching is often more useful than numeric results. These comments may help 
explain numeric results from SET and/or complement mean scores. 

(c) Focus Group Interviews are most effectively conducted with groups of students at the middle or 
end of the course in a semi-structured group interview format. These interviews allow a deeper 
analysis of all issues relating to ways to improve the course and allow the interviewer to probe for 
further clarification and meaning. They are considered a particularly valuable strategy to discuss 
results with students in more detail. Students’ views on content, assessment and delivery can be 
further explored and suggestions for improvement can be explained and/or challenged. The 
interviewer can either be the teacher or another tutor or colleague if greater anonymity and objectivity 
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is preferred. The focus groups should ideally consist of about six to eight students. The disadvantage 
is the whole student group cannot be interviewed due to time constraints. While student perceptions of 
course strengths and weaknesses are valued, it must be realised that students may not have a full 
understanding of pedagogy or content and interviews should be complemented with other forms of 
evaluation. Any weaknesses recognised by students can be challenged in this format where 
appropriate. 

Furthermore, in Australia, there have been some unique and useful developments in measuring and 
improving teaching and learning in the tertiary education sector. The software resource known as 
CEQuery is available to Australian universities at no cost. University staff can investigate the opinions 
of current and past students through the examination of feedback collected from completion of a 
variety of instruments (e.g. the CEQ). The process of examination is facilitated as comments are 
organised into themes and sub-analysis can extend to a range of background variables such as 
graduation year, field of study and gender. 

Peer Evaluation of Teaching 


Peer evaluation of teaching is an important source of feedback but is rarely used by academics. 
Centra (1987) has previously found that academics are more likely to ask a colleague to review a 
research manuscript than review a course outline or even visit a class. He suggested that academics 
may believe teaching is personal, observations are time consuming and stressful, research is more 
important and teaching improvement is not a high priority. However, just as peer review and 
discussion of research is essential, teaching can benefit from similar processes and opportunities 
(Paulsen, 2002) and can suitably complement other forms of evaluation. Lomas & Nicholls (2005) 
believed peer review of teaching should be viewed as a quality enhancement rather than a quality 
assurance process and can lead to considerable professional development of teachers. 

The value of peer evaluation has been discussed in the literature (Centra, 1987; Osborne, 1998). 
Peers generally have the contextual and content expertise to make meaningful judgements on a range 
of teaching processes (Paulsen, 2002). Peers may be in a better position to judge the quality of 
teaching in terms of subject matter knowledge, curriculum development, materials, course design, 
delivery and assessment. Students are usually only able to make informed judgements about delivery 
and assessment (Paulsen, 2002). When student feedback is complemented with peer advice, the 
combined effect on improvement is powerful (Murray, 1997). 

It is important that peer evaluation does not solely consist of classroom observations. Peer evaluation 
of teaching can occur both inside and outside the lecture theatre through both classroom observation 
and discussions of teaching philosophies and practices and shared critical reflections. Notably, peers 
can be involved in scholarly conversations about teaching. Evaluations conducted outside of the 
classroom can involve discussions around all aspects of teaching including assessment, teaching 
strategies, curriculum development, teaching materials, philosophies, strengths and areas for 
improvement. These conversations involve collaboration, reflection and mutual learning (McCormack, 
2003). Information collected qualitatively may allow richer and more useful, meaningful feedback 
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improve teaching practices (Osborne, 1998). It is also useful to ask both junior and senior colleagues 
to evaluate courses and teaching who may offer different perspectives and ideas. The reviewer should 
not always be a close colleague. 

It is important that the purpose of the evaluation be made explicit and to contextualise the process in 
terms of professional learning, growth and improvement. It is likely that the peer review process will be 
enhanced if explicit criteria are decided upon prior to evaluation. Furthermore, it is desirable for more 
than one peer to be involved and that each peer observes and/or evaluates on more than one 
occasion. The colleague should meet beforehand to discuss all aspects of teaching and decide upon 
the focus, purpose and process of evaluation. A useful strategy is for two teachers to evaluate each 
other on various occasions and use the experiences as a collegial and collaborative effort to improve 
teaching and course quality. 

In courses where other tutors are used, feedback can be collected regarding the quality of the course 
delivery and content and general or specific areas of improvement. This information can be particularly 
helpful to gain insight into the course from another teacher’s perspective. A tutor evaluation pro forma 
which examines a number of key areas relating to activities, assessment, materials etc can also be 
made available for tutors to complete at the end of the course. Open-ended questions can be asked 
regarding: What worked well? What didn’t go well? Suggestions for improvement for next year? 
Similarly, items can be developed to obtain numeric values for various aspects of the course. For 
example: 

Assessment tasks aligned properly with particular learning objectives SD D N A SA 

Self-Evaluation of Teaching 


Self-reflection can be used a number of ways including reflective journals, checklists, statements of 
teaching philosophy. Flowever, it is important that reflections are formally documented. A checklist 
could be used to rate perceptions of strengths and weaknesses of the course and teaching which may 
then be compared with other sources of evaluation. The self-reflection could also be a summary of all 
other sources of information to draw conclusions regarding course quality or elements of teaching that 
can be improved. A self-evaluation pro forma can be used which incorporates an analysis of a range 
of course and teaching factors. Open-ended questions can also be asked regarding What worked 
well? What didn’t go well? Suggestions for improvement for next year? Similarly, items can be 
developed (example above) to obtain numeric values for various aspects of the course and compared 
with results from other tutors (where appropriate). 

The quality of each course in terms of content and delivery can be reviewed at the conclusion of each 
semester, making note of a number of key evaluative considerations. For example, the 
appropriateness of certain activities, the quality of assessment tasks submitted, quality of teaching 
materials etc. Comments can be recorded on a course evaluation page to list activities that were 
perceived to be successful and comment on course effectiveness with a view to improving the course 
for the next year. Formative evaluative data in terms of anecdotal comments can also be collected 
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throughout the course after each lecture. Teaching portfolios are also considered a major component 
of reflective practice and teaching improvements (Saroyan & Amundsen, 2001), but the discussion of 
this method is beyond the scope of this paper. 

Other Components of Evaluation 

Mid-course evaluations are a particularly valuable form of formative assessment of the effectiveness 
of the course and individual teaching. Any of the sources of feedback described above can be utilised 
to examine positive and negative aspects of the course and explore suggestions as to how the course 
can be improved. Mid-course evaluations are particularly beneficial as changes can be made during 
the course rather than waiting until the end. Results of the mid-course evaluation can be made explicit 
to students who appreciate being involved in course decisions and that the teacher is concerned about 
the effectiveness of the course. 

Closing the Feedback Loop 

It is clear that to improve as a teacher one needs to evaluate their teaching. However, a teacher’s 
response to evaluative data is critical to the improvement of performance as the information collected 
does not automatically lead to improvements in teaching. For example, previous research has 
indicated that the use of student questionnaires does not improve the overall quality of teaching 
(Kember et al, 2002). More important than the numerical results is reflection by the teacher on what 
has been learned from the results. Reflective practice requires thinking carefully about aspects of 
courses/teaching and moving back and forth between thinking and experience and considering the 
future (McCormack, 2003). A teacher must use the information obtained to make needed changes. It 
is critical that the information collected first be summarised, taking into account the context, students 
and source of information. The extent of agreement and disagreement between sources and 
additional insights each source offers must be examined, along with a focus on the main strengths and 
weaknesses of the teaching and course. 

The extent to which feedback impacts on teaching practices is commonly referred to as ‘closing the 
loop’. There is a need to 'close the loop' to make sure that feedback sought from a variety of sources 
is used to maintain standards and to improve performance. 

The CIF provides a template for the application of an action inquiry framework to document not only 
the evaluative process, but also evaluative outcomes. Evaluation is described as a cyclical process 
that is based on an action inquiry framework. McCormack (2003) outlined the action inquiry process 
as a strategy to improve teaching practice and involves the following steps: 

1. collect information from a variety of sources using many approaches (act/observe) 

2. interpreting this information through reflection (reflect) 

3. Outline personal learning points from the reflection (plan) 

4. acting on these points (act). 


Journal of University Teaching and Learning Practice - Vol 5/2, 2008 





The Course Improvement Flowchart: A description of a tool and process for the 

evaluation of university teaching 

Philip Morgan 


Throughout this process, the C/F template can be used to document key recommendations emanating 
from specific sources of feedback. Appropriate use of this information requires an analysis and 
interpretation of the recommendations so that an action plan for course improvement is written. An 
action plan can bring together the recommendations into a proactive statement of intent. The action 
plan represents a combination of evaluative information and reflective interpretations to develop 
action-oriented tasks as a focus for future delivery. 

Credibility and Accountability of the CIF 

Two important aspects embedded in the CIF process are (i) ensuring students as key stakeholders 
are informed about the outcomes and action plan of the evaluation and (ii) that teachers are 
accountable to a colleague that this process has been undertaken and discussed. 

(i) It is critical that evaluation does occur in a two-way manner. Teachers who value student feedback 
may improve their own teaching and the quality of student feedback over time by increasing student 
motivation to participate in evaluation activities (Hughes, 2002). After taking into account all feedback 
from multiple sources of evaluation, all students should receive information regarding the impact of the 
evaluative process, otherwise their future motivation for participating in evaluations may be reduced 
(Hughes, 2002). Spencer and Schmelkin (2002) reported that students are keen to complete 
evaluations and are not worried about reprisal but believe their results are not examined or valued by 
faculties or administrators. Ballantyne (1999) found that students appreciated feedback from student 
evaluations indicating a concern and respect for students which is inherent with effective teaching. 
Students in this study really appreciated that their ideas were considered and made them think the 
teacher was approachable and concerned about their learning. Students felt more empowered and 
their beliefs were valued. 

As such, students should be told of specific changes made in response to their feedback. These can 
be achieved a number of ways including (1) emailing previous students or posting on student websites 
such as Blackboard about the specific impact of their evaluation on the new course offering; (2) 
reviewing changes made to the course in beginning lectures with new students to help them 
understand that the feedback and opinions from last year’s students were valued and influential; (3) 
the use of ‘your view counts’ posters which explicitly and publicly detail changes made on the basis of 
student feedback. 

(ii) To improve the chances of course improvement occurring, Moses (1986) believes consultation 
with a colleague is essential. Discussion with a colleague about the process and outcomes of 
evaluation allows a teacher to explain some of their personal reactions to evaluative data and seek an 
opinion regarding the appropriateness and value of the action plan (McCormack, 2003). Use of the 
CIF is much more effective when a colleague reviews the final draft and signs the flowchart. This 
process increases accountability and may also provide an additional perspective on the value and 
accuracy of recommendations and the action plan. The CIF can then be placed in a teaching portfolio. 
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Future Research 


Universities need to develop a greater understanding of what aspects of students’ learning 
experiences maximise learning, engagement and retention in different degrees. As such, any data 
collected from quantitative and qualitative evaluation need to be validated to establish the most critical 
factors in achieving positive student outcomes. For example, students could be asked to judge 
aspects of their university life in terms of both performance and importance. A key benefit from this 
more useful approach is that resources to improve practices can be more appropriately targeted at 
areas that are ‘performing’ poorly but recognised as of high ‘importance’. The current CEQ does not 
provide this data. Therefore, research should be conducted to help validate the 
importance/performance mix with successful graduates. 

To determine whether use of the CIF positively influences key student outcomes, research is required 
to test (a) whether the process is feasible and the experience worthwhile from a staff and faculty 
perspective and (b) whether the CIF as an evaluation tool impacts positively on student outcomes. To 
this end, a key question is to what extent systematic and comprehensive evaluative practices lead to 
improved student outcomes such as learning and satisfaction. Furthermore, each of the data collection 
methods for evaluation need to be studied more closely to establish both staff and student 
perspectives on the key advantages and disadvantages. For example, it is important to examine how 
students respond to feedback from teachers relating to the information they provide in various subject 
evaluations. A mixed-mode methodological approach to these research issues would provide a unique 
insight into strategies to improve the quality of teaching at universities. 

Conclusion 


Good teachers are interested in identifying what aspect of their teaching and course are strengths and 
which are weaknesses. Flowever, judgments of teaching effectiveness need to be made only after 
careful consideration of ways to measure teaching and course quality. In order to improve teaching, 
teachers should learn from students, colleagues and themselves by evaluating what they do and their 
impact. Comprehensive evaluation involves collecting information form multiple perspectives as each 
source on its own has limitations. The CIF provides a framework and tool to guide teachers through 
the evaluative process and facilitate the development of an action plan and valued outcomes from 
evaluation. There is no doubt that the evaluation process described throughout this article takes time 
and effort to appropriately implement. Flowever, the resultant improvements that may be achieved in 
teaching and course quality may lead to more satisfying teaching experiences, improved student 
outcomes and heightened professional growth as a teacher. 
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Method 


Results 
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Numerical 
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2. Open- 
ended 
comments 



3. Focus 
Group 
Interviews 



6. Tutor 
Course 
Evaluation 


7. Self¬ 
reflection 
Evaluation 


The average score for items was 4.34. Scores 
suggested students believed course delivery was 
(excellent and lectures stimulating. Improved mean 
scores for all items from previous 3 years. Strengths 
Iwere seen in enthusiasm, knowledge, preparation & 
'levels of respect shown to students. Lower scores for 
items relating to assessment from last year. 


Students were particularly pleased with the course 
and complimentary about many aspects including 
delivery, support, level of enjoyment & amount of 
knowledge and skills learnt to help them in their 
future careers. Suggestions centred on developing 
tutorial overviews and making these available on 
Blackboard. The assessment weightings and timing 
were criticised. 


Suggested developing a resource book and handout 
for each tutorial at the time of delivery. Suggested 
that a tutorial about accessing quality resources (esp. 
online) would be valuable. Believed the assessment 
task needed to be more relevant and the exam 
should not be worth 50%. Students would like a 
shared file set up on Blackboard for uploading and 
sharing assignments with other students. 


Numerical scores for the course evaluation aligned 
closely with scores for teaching. Availability of course 
resources and materials was an area in need of 
improvement. Assessment needs to more clearly link 
with outcomes and modified to improve professional 
relevance. 


Advice given regarding improving assessment 
practices including developing appropriate marking 
rubrics and accessing resource material. Suggestions 
around improving fairness in the assessment of 
group work and the use of a tool to encourage shared 
workload. Also look to vary teaching strategies in 
tutorials and aim to encourage all students to join in 
class discussions rather than 1 or 2 dominating. 


Students really enjoyed most activities and found 
tutorials worthwhile. Marking consistency meetings 
really helpful. Need more background information to 
deliver 1 st two tutorials. Need to delete some 
activities from weeks 4, 5 and 7. Believed tutorial 3 
was boring and difficult to teach. 

Tutorial 3 was too teacher directed. Assessment 
marking criteria was not detailed enough. Students 
desired greater choice in the assessment process. 
Students also required a fair amount of prompting 
throughout practical tutorials. Look to condense 
tutorial 1 and 2 and incorporate more student centred 
activities into tutorial 3. Practical tutorials were very 
well received - need readings to support. 



Fieure 1 The Course Improvement Flowchart 


COURSE IMPROVEMENT FLOWCHART: 


Enter Course Name and Code here 


Recommendations 

• Continue to think of ways to provide 
students with information about resources 
to support their teaching. Improvements 
can still be made in this area, despite 
students’ high level of satisfaction with this 
aspect in student evaluations. 

• Look to develop practical tutorial 
handouts and make available on 
Blackboard. 

• Think of ways to improve quality of 
resource booklet i.e. accessibility, 
presentation. 

• Reassess unit plan assignment. 

• Incorporate tutorial handout for all 
tutorials inclg. outline, aims, questions. 

• Look to further develop tutorials for 
activity 2. Link this activity and include 
manuscript as prescribed background 
reading. 

• Change assessment weighting and 
delete part 1 and focus on integrated 

.learning as assessment. 


Recommendations 

• Develop a marking rubric for unit plan to 
help students (particularly as most had not 
planned a unit of work before). Consider 
letting students select strand and stage for 
planning. Develop group work guidelines 
and template. 

• Include tutors as teaching assistants on 
Blackboard. 

• Re-develop tutorial in key weeks deleting 
activities suggested by tutors and 
increasing student centred approaches 

. • Develop summary sheet for activity 2. 


Action Plan for Course 
Improvement 

• Develop ‘Resources for Supporting 
Teaching’ handout and provide to 
students at start of course. 

• Develop practical handout worksheets 
that can be uploaded on Blackboard. 
Encourage students to bring a clipboard 
to practical tutorials and fill in answers 
and list suggestions during tutorials. 

• Do not include students’ original unit 
plans in resource booklet. Provide 
individual feedback and ask students to 
amend their units based on lecture 
feedback and resubmit for file share 
upload. The booklet will be a more 
valuable future teaching resource. 

• Develop marking rubric for unit plan 
and change assessment requirements 
to give students’ choice in unit plan task. 

• View DVD teaching resource week 
before tutorial or place in library for 
required viewing. 

• Find additional research articles for 
tutorials 1 and 2 and upload on 
Blackboard. Re-develop tutorial content 
and structure for weeks 1-3 and modify 
content based on tutor 
recommendations. 

• Make summary sheets for activities 
available for students and tutors 
including links to evidence and theories. 


Recommendations 

• Reshape tutorial 1, 2 and 3 so students 
are more familiar with concept and practice 
of assignment and theories/concepts. 

• Marking rubric should be considered as 
should more student centred activities. 

• Incorporate required readings to give 
students background information before all 

.practical tutorials. 
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